Skip to content

Comments

fix(loop): improve validation for greenfield builds#177

Closed
rubenmarcus wants to merge 1 commit intomainfrom
fix/loop-validation-improvements
Closed

fix(loop): improve validation for greenfield builds#177
rubenmarcus wants to merge 1 commit intomainfrom
fix/loop-validation-improvements

Conversation

@rubenmarcus
Copy link
Member

Summary

Fixes loop failures when building new projects from scratch (e.g., "build a React todo app"). Three root causes addressed:

  • Circuit breaker trips too early: The breaker counted consecutive validation failures across task boundaries. Now resets when a task completes (forward progress signal)
  • Package manager hardcoded to npm: All validation/init/wizard code assumed npm. Now auto-detects pnpm/yarn/bun from lockfiles and packageManager field
  • No warm-up period: Validation ran on every iteration, even Task 1 "Project Setup" where tests can't possibly pass. Now auto-skips validation for first ~50% of tasks on greenfield builds (configurable via --validation-warmup)

Changes

  • src/loop/executor.ts — Circuit breaker reset on task advancement + validation warm-up guard
  • src/loop/validation.ts — Use detected package manager in fallback path
  • src/commands/init.ts — Use detected PM in project detection + AGENTS.md generation
  • src/commands/run.ts — Use detected PM for dev commands + auto-detect greenfield warm-up
  • src/wizard/spec-generator.ts — Use detected PM for generated validation commands
  • src/cli.ts — Add --validation-warmup CLI flag
  • src/utils/package-manager.tsNEW package manager detection utility
  • src/utils/__tests__/package-manager.test.tsNEW 13 tests for PM detection

Test plan

  • pnpm build passes
  • pnpm test:run passes (156 tests, 13 new)
  • Manual test: run ralph-starter with --validate on a greenfield project

🤖 Generated with Claude Code

- Reset circuit breaker when tasks advance (prevents false positives
  during multi-task greenfield builds where early tasks can't pass tests)
- Add package manager detection: auto-detect pnpm/yarn/bun from lockfiles
  and packageManager field instead of hardcoding npm
- Add validation warm-up: skip validation until enough tasks are done for
  greenfield builds (auto-detected, configurable via --validation-warmup)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rubenmarcus rubenmarcus added the candidate-release PR is ready for release label Feb 12, 2026
@github-actions
Copy link
Contributor

Issue Linking Reminder

This PR doesn't appear to have a linked issue. Consider linking to:

  • This repo: Closes #123
  • ralph-ideas: Closes multivmlabs/ralph-ideas#123

Using Closes, Fixes, or Resolves will auto-close the issue when this PR is merged.


If this PR doesn't need an issue, you can ignore this message.

@github-actions github-actions bot added bug Something isn't working core tests labels Feb 12, 2026
@github-actions
Copy link
Contributor

✔️ Bundle Size Analysis

Metric Value
Base 1705.88 KB
PR 1722.79 KB
Diff 16.90 KB (0%)
Bundle breakdown
156K	dist/auth
28K	dist/automation
4.0K	dist/cli.d.ts
4.0K	dist/cli.d.ts.map
16K	dist/cli.js
12K	dist/cli.js.map
424K	dist/commands
28K	dist/config
4.0K	dist/index.d.ts
4.0K	dist/index.d.ts.map
4.0K	dist/index.js
4.0K	dist/index.js.map
536K	dist/integrations
84K	dist/llm
660K	dist/loop
172K	dist/mcp
32K	dist/presets
92K	dist/setup
28K	dist/skills
392K	dist/sources
76K	dist/ui
84K	dist/utils
328K	dist/wizard

@rubenmarcus
Copy link
Member Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7aea4a25c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +48 to +50
if (script === 'test') {
return { command: pm, args: ['test'] };
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use bun run for Bun test scripts

getRunCommand maps every test script to <pm> test, but for Bun that command invokes Bun’s built-in test runner, not the package.json script (bun --help describes test as “Run unit tests with Bun” while bun run <script> is for package scripts). In Bun projects that define scripts.test (for example vitest), validation and generated commands will run the wrong command and can report misleading pass/fail results.

Useful? React with 👍 / 👎.


// Auto-detect greenfield builds: skip validation until enough tasks are done
const isGreenfield = taskCount.total > 0 && taskCount.completed === 0;
const autoWarmup = isGreenfield ? Math.max(2, Math.floor(taskCount.total * 0.5)) : 0;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Prevent warm-up from skipping all validation on small plans

The auto warm-up floor of 2 means a one-task plan gets validationWarmup = 2, but validation only runs once completedTasks >= validationWarmup; with a single task that threshold is unreachable before completion, so --validate is effectively disabled for the whole run. This allows single-task greenfield runs to finish without any test/lint/build gate.

Useful? React with 👍 / 👎.

@rubenmarcus
Copy link
Member Author

Superseded by #188 — all changes consolidated into a single PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working candidate-release PR is ready for release core tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant